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Abstract 

This article explores and summarizes trends in research and scholarship over the last decade (i.e., 
1998-2007) for students completing dissertations and theses in the area of distance education. 
The topics addressed, research designs utilized, and data collection and analysis methods used 
were compiled and analyzed. Results from this study indicate that most of the distance education 
research conducted by graduate students in this period of time has been descriptive, often 
addressing the perceptions, concerns, and satisfaction levels of various stakeholders with a 
particular distance education experience. Studies of this type typically used self-report surveys 
and analyzed the data using descriptive statistics. Validating the concern of many distance 
education scholars, there was a lack of graduate student research aimed at developing a theory 
base in distance education. On a positive note, projects directly comparing distance education 
with traditional face-to-face classrooms to determine the merit of specific programs declined 
significantly in 2007 as compared to 1998. This result might indicate that distance learning is 
becoming accepted as a viable and important educational experience in its own right. Another 
encouraging finding was the decreased emphasis on studies focused on technology issues, such as 
those analyzing the quality of distance education technology and questioning educators’ ability to 
provide an acceptable technology-enabled distance learning experience. 
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When considering the general state of distance education research, an important starting point is 
to examine what is published in scholarly journals and to conduct a review of theses and doctoral 
dissertations (Moore & Kearsley, 2005). Several articles over the past decade have chronicled the 
research trends of studies published in major distance education journals (e.g., Berge & 
Mrozowski, 2001; Lee, Driscoll, & Nelson, 2004; Ritzhaupt, Stewart, Smith, & Barron, 2010; 
Zawacki-Richter, Baecker, & Vogt, 2009). This article explores and summarizes trends in 
research and scholarship over the period of 1998-2007 for students completing dissertations and 
theses in the field of distance education. More specifically, the topics addressed, research designs 
utilized, and data collection and analysis methods used were compiled and analyzed. 

General State of Distance Education Research 

There is little doubt that distance education is an innovative and expanding field (Allen & 
Seaman, 2007). In 1995 only one-third of the institutes of higher education in the United States 
offered distance education courses (Lewis, Snow, & Farris, 1999). The most recent national 
study (2006-07) on distance education sponsored by the Department of Education indicates that 
“two-thirds (66%) of 2-year and 4-year Title IV degree-granting postsecondary institutions 
reported offering online, hybrid/blended online, or other distance education courses” (Parsad & 
Lewis, 2008, p. 2). For a variety of reasons, distance education and online learning are appealing 
to students, teachers, and administrators in many fields. But even with this level of acceptance 
and use, many researchers acknowledge that unless the amount and quality of distance education 
research and scholarship are improved, substantial improvements in teaching and learning are 
unlikely (Lee, Driscoll, & Nelson, 2004; Moore & Kearsley, 2005). 

Naidu (2005) observed that the majority of distance education research has been descriptive (i.e., 
studies that describe how or what is being done in a case study context) and kindly suggested that 
the rigor and quality of much of this research is suspect. While high-quality descriptive research 
has its place and contributes to the development of a working knowledge of important aspects of 
the field, some argue that distance education must develop new scientific models using more 
rigorous research methodologies (Tallent-Runnels et al., 2006). 

Another common type of research used to study distance education programs and initiatives is 
evaluation research, which examines the effectiveness of distance education practices (Moore & 
Kearsley, 2005). Often, effectiveness evaluations are based on a comparison with traditional 
face-to-face classrooms (Gaytan, 2007; Tucker, 2001). The typical criteria for measuring the 
effectiveness of distance education instruction focus on analyses of student achievement in, 
attitude toward, and satisfaction with the learning experience (Phipps & Merisotis, 1999). Critics 
of this practice point out the poor methodological design of some of these comparison studies and 
the questionable quality of assessment instruments used to gather comparison data; they also 
suggest that studies simply comparing faculty and student perception of and satisfaction with 
distance learning and traditional models of face-to-face instruction are rather weak evidence of 
value (Beaudoin, 2004; Bernard, Abrami, Lou, & Borokhovski, 2004; Meyer, 2002; Tallent- 
Runnels et al., 2006). More fundamentally, while most studies show distance education to be as 
effective as traditional education (Meyer, 2004; Russell, 1999; Saba, 2000; Simonson, 2002; 
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Zhao, Lei, Yan, Lai, & Tan, 2005), the need to validate the importance and viability of distance 
education based on comparisons with face-to-face learning experiences seems to expose a deep- 
rooted insecurity within the distance learning community—a fear that distance education is 
regarded as a somewhat substandard and less valued educational practice. This phenomenon has 
prompted calls for more formative evaluation practices to address concerns regarding the need for 
(1) improving the distance education experience, (2) establishing acceptable principles of best 
practice, and (3) developing standards of quality by which distance education practices can be 
judged (Beaudoin, 2004; Garrison & Anderson, 2003; Meyer, 2004; Sherry, 2003). 

Finally, while it should not be assumed that quality distance education research does not exist 
(Meyer, 2002), many distance education scholars express concern regarding the perceived 
emphasis on the pragmatic rather than the theoretical. They point out the apparent inadequacy of 
research aimed at establishing a solid theory base from which distance education can develop 
(Beaudoin, 2004; Garrison & Anderson, 2003; Saba, 2003; Moore & Kearsley, 2005). New 
scholars typically learn to conduct research in graduate school as they complete thesis and 
dissertation projects. For this reason, an analysis of research topics and methods in graduate 
schools promises to provide an important perspective and update on the state of research in the 
field. 


Research Methods 

This study used content analysis techniques to determine trends in research topics or purposes, 
research designs, and types of data collection and analysis methods. A thematic analysis was 
employed to determine the most frequently addressed topics and most commonly used designs 
and methods in order to explore changes in these aspects of graduate student research in distance 
education for the period of 1998-2007. 

Manuscript Selection Criteria and Process 

Moore and Kearsley (2005) point out the difficulty researchers have in accessing all the relevant 
graduate student research on the topic of distance education. Internet technologies make this task 
possible, but some studies are not labelled as distance education research per se, and many 
manuscripts have been submitted with abstracts only. For this study, abstracts alone were 
insufficient for the desired analysis; full-text manuscripts were needed. In addition, the time and 
effort involved in reading and categorizing a decade’s worth of available research manuscripts 
presented a daunting task. This study sampled manuscripts at three points in the last decade (i.e., 
1998, 2002, & 2007) to uncover any trends that may exist. 

The sample used in this study includes all full-text English doctoral dissertations and master’s 
theses located using the descriptor distance education submitted to the ProQuest Dissertation and 
Theses Database (PQDT) in 1998, 2002, and 2007. PQDT (formally known as UM1) is a 
commercial database housing a searchable archive of published dissertations and theses (see 
proquest.com) . This database provided a suitable pool of graduate student research from North 
America from which we could study this issue. A representative from ProQuest disclosed to the 
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authors that PQDT receives 97.2% of all dissertations and theses from research universities in the 
United States (276 of 284) and 87.2% (41of 47) of those from Canadian research universities 
(personal correspondence, May 17, 2010). 

A keyword search using the general search criteria for the phrase distance education was 
performed. The thesaurus for the Education Resources Information Center (ERIC), sponsored by 
the United States Department of Education, added distance education to its controlled vocabulary 
on October 24, 1983. No similar phrases or terms related to distance education were included in 
the search criteria for manuscripts; however, the following related terms are referenced to 
distance education as part of the ERIC thesaurus: asynchronous communication, blended 
learning, computer-mediated communication, continuing education, correspondence schools, 
educational radio, educational television, electronic learning, extension education, external degree 
programs, geographic isolation; handheld devices, home study, independent study, laptop 
computers, lifelong learning, mass instruction, nontraditional education, online courses, open 
universities, outreach programs, part-time students, synchronous communication, 
telecommunications, telecourses, virtual classrooms, virtual universities, and web-based 
instruction (see www.eric.ed.gov) . 

In 1990, ERIC had 1,260 academic submissions associated with the controlled vocabulary 
distance education. By 1995 the number of citations in this category had increased to 2,709, and 
by the time of this writing, the number had increased to just under 12,000. 

Manuscript Coding 

Each manuscript selected for analysis was read and coded by two of the seven graduate students 
who participated in the manuscript coding process. All raters, who were paid by the hour, were 
trained in the coding process, and random quality checks were performed to ensure a satisfactory 
level of coding, with training updates provided as needed. Each manuscript was categorized on 
the general topics addressed in the study, the research designs utilized, and the data collection and 
analysis methods used. Initial inter-rater reliability was determined; however, all discrepancies in 
ratings were arbitrated by an independent third rater to establish a definitive final count in each 
area. Many individual manuscripts addressed more than one topic or utilized multiple data 
collection and analysis methods. All principal topics addressed and methods used in each study 
were included in the count. Results of the classifications for each of the four areas were 
compared across years. 

Classification of Coding Categories 

Categories for coding were determined using an a priori approach. Topic categories largely 
follow those identified in a similar study conducted by Lindsay, Wright, and Howell (2004). 
Table 1 provides a summary of the topic categories with a description of category contents. 
Quantitative research designs were identified from research texts; however, qualitative research 
designs do not share the same degree of specificity and therefore were generally classified as 
qualitative survey research (i.e., surveys with open-ended questions), ethnographic studies, or 
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narrative phenomenological studies (see Table 2). Data analysis techniques were identified from 
research texts; however, since qualitative analysis methods were only generally described by 
student researchers, they are categorized together. Qualitative analysis usually included 
segmenting (organizing) data from open-ended surveys, interviews, and observations then 
describing patterns found in the responses or observations. 

Table 1 

Research Topic Descriptions 


Topic 

Description 

Student issues 

Issues students face or perceptions they express 
concerning distance education 

Faculty issues 

Issues faculty/instructors face or perceptions 
they express concerning distance education 

Pedagogical issues 

Teaching and learning issues involved with 
distance education 

Instructional design 

Issues involving the design of instruction for 
distance education 

Methods testing 

Test of a distance education method, including 
comparisons of the achievement, attitudes, or 
satisfaction levels of those involved 

Instructional methods description 

Description of a specific implementation or the 
current status of a distance education program 

Distance education theory 

Research involving DE theory or theory-based 
conceptual framework of distance education 

Economic issues 

Research looking primarily at the economics of 
distance education 

Govemance/administration issues 

Studies concerned primarily with policy issues 
and administration of DE programs 

Technology issues 

Technology issues related to distance education 

Scalability 

Possibility or feasibility of scaling distance 
education to larger audiences 

Historical foundations 

Historical perspectives and foundations of 
distance education 

International perspectives/transferability 

Context of international venues and transfer of 

distance education to different audiences 
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Table 2 

Research Design Descriptions 


Design 

Description 

Random controlled trial 

Comparison groups determined through 
randomization to explore causal effect 
relationship 

Quasi-experimental 

Comparison groups determined with non- 
random methods to explore causal effect 
relationship 

Casual comparative 

Comparison of groups that were pre¬ 
determined—could not be randomly assigned 
(also called ex post facto design) 

Correlational 

Process of determining the relationship or 
degree to which relationships exist between two 
or more variables in the study 

Descriptive quantitative survey research 

Surveys designed to understand/describe DE 
situation (predetermined variables, close-ended 
items) 

Descriptive qualitative survey research 

Surveys (open-ended items) designed to 
understand/describe DE situation (specific 
variables typically not predefined) 

Ethnographic 

Descriptive analysis of cultural patterns or 
perspectives for identifiable groups of people 

Narrative phenomenology 

Narrative of individual(s) experiencing a 
specific phenomenon or event 

Case study 

Account of a specific instance, generally 
descriptive 

Other 

e.g., literature reviews 


Findings and Discussion 

Certainly this experience has demonstrated for the researchers the variability in the quality of 
current graduate student research. This study does not, however, attempt to judge the quality or 
appropriateness of the methods graduate students utilized to conduct distance education research. 
This analysis is primarily descriptive with the intention of understanding what topics graduate 
students studied and what methods were employed in their research. 

Trends in Research Topics 

Table 3 presents the distribution of research topics addressed each year. Approximately 100 
research papers were extracted for each of the years sampled in this study. The sample includes 
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all manuscripts fitting the selection criteria each year. The “percentage of total” columns in the 
table do not add to 100% as some of the studies addressed more than one topic. For example, 
some papers considered both student and faculty issues in the same study. The frequency counts 
provided represent how often specific purposes or general topics were addressed by graduate 
student researchers. 

Table 3 

Research Topic Distribution by Year 



2007 


2002 


1998 


count 

%of 

total 

count 

%of 

total 

count 

%of 

total 

Student issues 

32 

29.4 

21 

21.0 

17 

17.2 

Faculty issues 

21 

19.3 

21 

21.0 

12 

12.1 

Pedagogical issues 

9 

8.3 

14 

14.0 

13 

13.1 

Instructional design 

10 

9.2 

7 

7.0 

12 

12.1 

Methods testing 

18 

16.5 

21 

21.0 

17 

17.2 

Instructional methods description 

8 

7.3 

10 

10.0 

14 

14.1 

Distance education theory 

2 

1.8 

2 

2.0 

1 

1.0 

Economic issues 

1 

0.9 

3 

3.0 

0 

0.0 

Governance/administration issues 

7 

6.4 

11 

11.0 

9 

9.1 

Technology issues 

3 

2.8 

6 

6.0 

17 

17.2 

Scalability 

1 

0.9 

0 

0.0 

0 

0.0 

Historical foundations 

0 

0.0 

3 

3.0 

5 

5.1 

International Perspectives/transferability 

3 

2.8 

1 

1.0 

2 

2.0 

Other 

1 

0.9 

1 

1.0 

0 

0.0 

Total documents 

109 


100 


99 



Note: Initial inter-rater reliability 65% — all discrepancies in ratings were arbitrated for the final count 


Teacher, student, and administrative issues. 


Based on the research being conducted in this study, the data trends seem to suggest a fairly 
consistent research emphasis on student and faculty issues. These categories include topics that 
address the perceptions of stakeholders, i.e., their attitudes toward, satisfaction with, and thoughts 
regarding specific distance education experiences. Governance and administrative issues as 
research topics also fall into this general area of research. They typically follow a similar type of 
research design and, though less frequent, have been fairly consistent as topics of interest. 


Methods testing. 


Research that tests methods falls into the category of evaluation research often labelled as media 
comparisons due to the tendency of researchers to compare distance and traditional instructional 
practices. Testing distance education methods has also been fairly consistent as a purpose of 
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many studies. Yet while the frequency of methods testing studies has remained fairly consistent, 
the trend has moved away from comparisons with face-to-face classroom experiences. In 1998, 

12 of the 17 methods testing studies (71%) determined the effectiveness of the distance education 
initiative by a comparison with a traditional face-to-face learning experience; in 2007, this 
number dropped to 5 of the 18 studies (28%). 

Instructional design and pedagogy. 

Studies that have considered the design and pedagogy involved in distance education learning 
situations have also been fairly common, although graduate students’ interest in studying such 
topics seems to have declined slightly since 1998. 

Technology issues. 

An interesting trend in research topics is the decrease in studies addressing technology. 
Apparently, concern for whether distance education technology would be reliable or advanced 
enough to facilitate the demands of distance education has diminished considerably. While 
technology issues were a large concern in 1998, with 17 studies addressing this issue, only 3 
studies researched this topic in 2007. 

Research no-shows. 

Several areas of research seem to be of less interest to graduate students. Distance education 
theory is the most notable in the list of infrequently studied topics or purposes, along with 
economic issues, scalability, historical foundations of distance education, and studies involving 
an international perspective. To be fair, many students cited distance education theory, or in 
some way tested theory, in their studies. Every study analyzed in this sample included a literature 
review of some sort. But in this sample, only a couple of graduate student studies each year 
focused their research directly on theory development or exploration. 

Trends in Research Designs 

The frequency of various research designs utilized each year in graduate student research is 
reported in Table 4. 


49 


A Review of Trends in Distance Education Scholarship at Research Universities in North America, 1998-2007 

Davies, Howell, and Petrie 


Table 4 

Research Design Distribution by Year 



2007 


2002 


1998 


count 

%of 

total 

count 

%of 

total 

count 

%of 

total 

Random controlled trial 

8 

7.3 

3 

3.0 

0 

0.0 

Quasi experimental 

11 

10.1 

11 

11.0 

8 

8.1 

Casual comparative 

7 

6.4 

3 

3.0 

0 

0.0 

Correlational 

9 

8.3 

2 

2.0 

5 

5.1 

Descriptive quantitative survey research 

46 

42.2 

38 

38.0 

33 

33.3 

Descriptive qualitative survey research 

8 

7.3 

5 

5.0 

1 

1.0 

Ethnographic 

4 

3.7 

3 

3.0 

4 

4.0 

Narrative phenomenology 

3 

2.8 

3 

3.0 

1 

1.0 

Case study 

24 

22.0 

27 

27.0 

32 

32.3 

Other 

1 

0.9 

8 

8.0 

16 

16.2 

Total documents 

109 


100 


99 



Note: Initial inter-rater reliability 70% — all discrepancies in ratings were arbitrated for the final count 


As Naidu (2005) suggests, most student research seems to be descriptive. A pattern from the 
research studies analyzed in this sample indicates a strong and increasing reliance on survey 
research designs and case studies involving self-report evidence from stakeholders. The number 
of random controlled trials and casual comparative (i.e., ex post facto) designs increased, but 
consistently the method for establishing comparison groups was to select participants from 
existing groups or convenient samples (i.e., quasi-experimental designs). The number of studies 
using qualitative surveys has increased (i.e., predominantly surveys using open-ended items with 
no specific predetermined variables of interest), but the frequency with which qualitative designs 
have been employed remains fairly small and consistent. 

Trends in Research Data Collection and Analysis 

Tables 5 and 6 present the various data collection techniques and data analysis methods used 
most often each year in graduate student research. Since the predominant research design used in 
this sample involved survey research, it is understandable that the most commonly used data 
collection method involved surveys. More than half of the studies utilized a survey of some type, 
including both self-report surveys and attitudinal scales. Student researchers tended to use 
interviews as a principal source of qualitative data, although many qualitative studies used a 
variety of data collection methods, including surveys or analyses of existing documents and 
artifacts. 
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Table 5 

Data Collection Methods Distribution by Year 



2007 


2002 


1998 



count 

%of 

count 

%of 

count 

%of 



total 


total 


total 

Standardized assessments 

7 

6.4 

5 

5.0 

3 

3.0 

Researcher created assessments 

8 

7.3 

5 

5.0 

3 

3.0 

Existing test scores 

8 

7.3 

6 

6.0 

2 

2.0 

Quantitative survey instrument 

57 

52.3 

57 

57.0 

50 

50.5 

Qualitative survey instrument 

15 

13.8 

7 

7.0 

6 

6.1 

Observations 

1 

0.9 

8 

8.0 

8 

8.1 

Interviews 

39 

35.8 

41 

41.0 

38 

38.4 

Existing artifacts 

11 

10.1 

24 

24.0 

24 

24.2 

Other - literature review of other research 

0 

0.0 

1 

1.0 

9 

9.1 

Total documents 

109 


100 


99 


Note: Initial inter-rater reliability 83% — all discrepancies in ratings were arbitrated for the final count 



Table 6 







Data Analysis Technique Distribution by Year 








2007 


2002 


1998 



count 

%of 

count 

%of 

count 

%of 



total 


total 


total 

t-test 

22 

20.2 

18 

18.0 

9 

9.1 

Analysis of variance (ANOVA, ANCOVA) 

19 

17.4 

18 

18.0 

15 

15.2 

Multiple regressions 

7 

6.4 

3 

3.0 

1 

1.0 

Factor analysis 

2 

1.8 

1 

1.0 

0 

0.0 

Linear regression 

1 

0.9 

2 

2.0 

2 

2.0 

Correlations 

14 

12.8 

9 

9.0 

12 

12.1 

Exclusively/primarily descriptive statistics 

43 

39.4 

31 

31.0 

24 

24.2 

Non-parametric analysis (e.g., chi-squared) 

13 

11.9 

6 

6.0 

7 

7.1 

Qualitative analysis 

43 

39.4 

40 

40.0 

35 

35.4 

Other - personal interpretation 

1 

0.9 

3 

3.0 

18 

18.2 

Total documents 

109 


100 


99 



Note: Initial inter-rater reliability 78% — all discrepancies in ratings were arbitrated for the final count 


Most of the studies in this sample used some descriptive data analysis (e.g., frequencies & 
percentages). Studies identified specifically as using descriptive statistics were those that used 
this type of data analysis exclusively or predominantly. A large number of student researchers 
did use descriptive statistics as their main analysis tools. 
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In the studies from this sample, the qualitative data analysis methods were not described in 
specific detail; thus, qualitative data analysis methods were combined in the count for this study. 
Typical qualitative data analysis seems to have included segmenting or organizing data from 
open-ended surveys, interviews, and observations then describing patterns found in the responses 
or observations. Trends in the amount of qualitative data analysis being used seem proportionally 
similar to the number of qualitative data collection methods used. 

Of interest in this data set is the frequent use of quantitative statistical analysis techniques 
involving t-tests and ANOVA analysis. The use of such data analysis techniques seems high, 
given the data collection methods employed. One observation from the coding of manuscripts 
that might help explain this apparent inconsistency is that students often used these types of 
analysis to make comparisons in survey results based on disaggregated groups of respondents. 
While the appropriateness of this practice with survey data is suspect, given the type of data that 
surveys produce and the assumptions regulating the use of these analysis techniques (Reynolds, 
Livingston, & Willson, 2006), this is what was reportedly done, and it may help explain the 
disproportionate frequencies. 


Conclusions 

This analysis of dissertations and theses in the field of distance education provides a macro 
perspective that promises to inform future research and meta-analysis. One limitation of this 
study is that it does not include student research conducted outside of North America. Still, it is 
evident that during this past decade, while the number of dissertations and theses prepared in 
North America has remained fairly static, shifts in topics studied and research methods used have 
occurred. 

Consistent with Naidu’s (2005) observations regarding the types of research being conducted in 
the field of distance education at that time, this study found that over the past decade, most 
graduate level research has been descriptive. More often than not, graduate students’ research has 
addressed the perceptions, concerns, and satisfaction levels of various stakeholders with a 
particular distance education experience. These types of studies usually administered self-report 
surveys and analyzed the data using descriptive statistics. While there is value in conducting 
quality descriptive research, a lack of research addressing other important topics is evident. 

Validating the concern of distance education scholars regarding the lack of research intended to 
establish distance education theory (Beaudoin, 2004; Garrison & Anderson, 2003; Saba, 2003; 
Moore & Kearsley, 2005), this study found little graduate student research aimed at developing a 
theory base for distance education. Unfortunately, far too few studies explored new or challenged 
old theory. Factors that may help explain this finding include the challenges associated with 
conducting any type of grounded theory research. Many graduate students lack the experience, 
time, and resources needed to conduct this type of research. Additionally, they may be limited in 
their access to the participants and educational situations needed to rigorously explore and 
establish distance education theory. Unfortunately, the purpose for having graduate students 
conduct research is often to have them demonstrate their ability to conduct research rather than to 
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conduct groundbreaking research. Regrettably, these data suggest a lack of grounded theory 
research. It may be incumbent on research institutes that study distance education to encourage 
students to engage more in theory-based research. They might also consider more carefully the 
analysis methods used and the degree to which analysis techniques align with the data collection 
methods. 

On a more positive note, we were encouraged to see a notable trend away from instructional 
media studies that compare distance education with traditional instructional practices. Evaluation 
that involves methods testing has been a consistent incentive for conducting research; however, 
between 1998 and 2007 far fewer graduate research projects attempted to determine the merit or 
worth of the specific distance education practice by making explicit comparisons with traditional 
face-to-face learning environments. This decrease might indicate that we, as a community of 
researchers and perhaps society in general, are beginning to accept distance learning as an 
important and viable educational experience in its own right. 

Another encouraging finding is the decreased number of studies focused on technology issues, 
particularly concern about the quality of technology and the ability of distance educators to 
provide an acceptable technology-enabled learning experience. By most measures, the quality 
and availability of educational technology in schools has increased significantly as has the 
technological literacy of teachers and students (McMillan-Culp, Honey, & Mandinach, 2005; 
Russell, Bebell, O’Dwyer, & O’Connor, 2003). This progress has not eliminated technology 
problems, but those in a distance learning setting seem to have accepted that technology problems 
will occur, and they cope with the challenges associated with technology use when they happen. 
This is a potential topic for further research. 
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